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APPARATUS AND METHOD FOR ABSTRACTING MOTION PICTURE SHAPE 
DESCRIPTOR INCLUDING STATISTICAL CHARACTERISTICS OF STILL 
PICTURE SHAPE DESCRIPTOR, AND VIDEO INDEXING SYSTEM AND 

METHOD USING THE SAME 

5 ' 
Technical Field 

The present invention relates to an apparatus and 
method for abstracting motion picture shape descriptors 

10 having statistical characteristics of still picture shape 
descriptors, a video indexing apparatus using the motion 
picture shape descriptor abstracting apparatus and method, 
and a computer-readable recording medium for recording a 
program that implements the motion picture shape descriptor 

15 abstracting method- 

Background Art 

Increasing amount of video and audio data calls for 
20 technologies for retrieving and managing the data 
efficiently. One of these technologies is a multimedia 
indexing technique for abstracting indexing information 
representing multimedia data to be used for data retrieval 
and searching. 

25 Currently, with respect to a still picture, color 

histograms, shape descriptors and/or texture descriptors 
are used to abs"tract indexing information representing 
multimedia data, and for audio data, spectrum descriptors 
are used; With respect to a motion picture, motion 

30 information descriptors using motion vectors and/or orbit 
descriptors of objects are used. However, the descriptors 
of the conventional technologies' are not those descriptors 
used to dynamically index shape information of objects 
within a video. 

35 In addition, as one dynamic, indexing method for 

indexing the dynamic change of shape data, there is a 
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method- indexing the shape information of an object from the 
entire still pictures that compose a motion picture or from 
some representative still pictures by using the 
conventional still picture shape information indexing 
method. However, this method has a shprtcoming that the 
data storing and retrieving efficiency is poor, because the 
amount of indexing information is increased, as the number 
of still pictures' used for abstracting shape data is 
increased. 

Disclosure of Invention 

It is, therefore, an object of the present invention 
to provide an apparatus and method for abstracting motion 
picture shape descriptors by abstracting still picture 
shape descriptors from the still pictures of an object that 
compose a motion picture and abstracting motion picture 
shape descriptors having statistical characteristics from 
the abstracted still picture shape descriptors to use them 
as video indexing information, a video indexing system 
using the motion picture shape descriptor abstracting 
apparatus and method, and a computer-readable recording 
medium for recording a program that implements the motion 
picture shape descriptor abstracting method. 

In accordance with one aspect of the present invention, 
there is provided a system for retrieving motion picture, 
comprising: a motion picture segmentation means for 
segmenting motion picture temporally; a motion picture 
shape descriptor . abstracting means for abstracting a motion 
picture shape descriptor from the segmented motion picture 
data; and a motion picture metadata storing means for 
storing the motion picture shape descriptor as metadata. 

Brief Description of Drawings 

The .above and other objects and features of the 
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present invention will become apparent from the following 
description of the preferred embodiments given in 
conjunction with the accompanying drawings, in which: 

Fig. 1 is a block diagram ■ showing a motion picture' 
5 shape descriptor apparatus and a motion picture retrieving 
system in accordance with an embodiment of the present 
invention*; 

Fig. 2 is a block diagram illustrating the motion 
picture shape descriptor abstracter of Fig. 1 in accordance 

10 with the embodiment of the present invention; 

Fig. 3 is a table showing the metadata stored in a 
motion picture metadata database for storing motion picture 
shape descriptors in accordance with the embodiment of the 
present invention; and 

^5 ^ig- 4 is a flow chart describing a method for 

abstracting motion picture shape descriptors in accordance 
with the embodiment of the present invention. 

Best Mode for Carrying Out the Invention 

20 

Generally, shape descriptors of objects for a still 
picture include outline-based shape descriptors and region- 
based shape descriptors. The present invention suggests 'a 
motion pictu:re shape descriptor, ' which refers to a 

25 descriptor obtained by abstracting shape descriptors, 
including outline-based shape descriptors or region-based 
shape descriptors, from the respective still pictures of 
objects composing a motion picture, and processing the 
abstracted shape descriptors statistically. . The 

30 statistically processed motion picture shape descriptors, 
i.e., statistical characteristic descriptors, have moment 
characteristics, such as mean and variance. 

Following is a process for abstracting motion picture 
shape descriptors in a statistical shape vector descriptor 

35 abstracter. 

The shape sequence of an input object is expressed as 
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SS={si, S2r S3^..., Sn}. Here, Sm denotes an m^^ shape. A 
sequence of still picture shape descriptor SD={sdi, sda, 
sd3,..-, sdn) is obtained with respect to each shape from 
the above shape sequence by using the conventional still 
picture shape descriptors, e,g., region-based ones or 
outline-based ones. Here, sdm is a still picture shape 
descriptor abstracted from an m^^ shape Sm- The still 
picture shape descriptor sdm is generally expressed as* an 
equation in the form of a vector, i.e.. Equation 1 below, 

sdm = {Sdiad), Sdin(2), SdmO),..., Sdmd)} 

Eq. 1 

wherein 1 denotes the number of elements that form the 
15 vector, and sdm(p) represents a p^^ element. 

The present invention forms a motion picture shape 
descriptor by using the sequence SD of the still picture 
shape descriptor and abstracting four shape descriptors (1) 
20 to (4), enumerated below. 



(1) Meian Shape Descriptor 

Mean shape descriptor sd^"" = {sd^''(l), sd^'''(2),' 
sd^^'O) , . . . , sd^'^d) } is abstracted as follows. An m^^ 
25 element sd^^{m) is the mean value of the m^^ element of each 
of n number of shape descriptors that forms S-D = {sdl, sd2, 
sd3, . , . , sdn}. It can be obtained based on Equation 2. 

sd^^(m) = to n sdi(m) ) /n Eq. -2 

30 . ' * 

(2) Variance Shape Descriptor 

The variance shape' descriptor sd""^"^ = {sd''^'^(l), sd''^'^{2), 
sd^^^O) , . . . , sd^'^^'d)} is abstracted as follows. That is, 
an m^^ element sd"^^"" (m) is a variance value of the m^^ 
35 element of each of n number of shape descriptors that form 
SD == {sdl, sd2, sd3,..,, sdn}. It can be obtained based on 
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Equation 3. . 

sd^"(m) = (2:1=1 ton (sdidn)- sd^^Cm) )2)/n/ (n-1) 

Eq. 3 

5 . " 

(3) Standard Deviation Shape Descriptor 

The standard "deviation shape descriptor sd^*"^ 
{sd="(l), sd^"(2), sd="(3),. sd^'^^d)} is abstracted as 
follows. That is, an m^^ element sd="(m) is a standard 
10 deviation value of the m*''' element of each of n number of 
■ shape descriptors that form SO = {sdl, sd2, sd3, . . . , sdn} . 
It can be obtained based on Equation 4 . 

sd=^^(m) = sqrt{2i=iton {sdi(m)- sd^^(m) ) 2) / (n-1) 
15 Eq. 4 

(4) Differential Shape Descriptor 

The differential shape descriptor shows the change of 
two consecutive shape descriptors in a shape descriptor 
20 sequence. The differential shape descriptor sequence DSD = 
{dsdi, dsd2, dsd3,.. dsdn-il can be obtained from the shape 
descriptor SD = {sdl,. sd2, sd3,..., sdn} based on Equation 

25 dsdr = (Sdr+i * Pr+l) (sdr * Pr) 

- ■ ■ Eq. 5 

wherein r is in the range of 0 < r- < n, and pr denotes 

a weight of an r*='' shape descriptor sdr, which can be 

30 obtained from a time rate of a shape represented by a shape 
descriptor occupying in the entire shape sequence. 

The mean shape descriptor, variance shape descriptor 
and standard deviation shape descriptor, i.e., (1), (2) and 
35 (3), are obtained from the differential shape descriptor 
sequence DSD = {dsdi, dsda, dsds, . • • / dsd„-i} and used for 
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abstracting motion picture shape descriptors. 

The motion picture shape descriptors, suggested in the 
present invention can use the above shape descriptors alone 
or a combination thereof. " The motion picture shape 
descriptors abstracted by using a combination of the shape 
descriptors can be expressed as: 

CSSD. = {cssdi, cssd2, cssdi, . . , , cssdi) . 

wherein cssdi is one of motion picture shape 
descriptors suggested in the present invention. 

If a still picture shape descriptor which is 
irrespective of the change in size and rotation is applied, 
a motion picture ■ shape descriptor also irrespective of the 
change in size and rotation is obtained. 

This method of abstracting a statistically processed 
motion picture shape descriptor can be used with respect to 
other still picture descriptors, such as still picture 
texture descriptor, other than the shape descriptors used 
in the embodiment of the present invention. Therefore, the 
technology of the present invention has an advantage that 
it can be generalized. 

Other objects and aspects of- the invention will become 
apparent from the following description of the embodiments 
with reference to the accompanying drawings, which is set 
forth hereinafter. 

Fig. 1 is a block diagram showing a motion picture 
shape descriptor apparatus and a motion picture retrieving 
system in accordance with an embodiment of the present ' 
invention. 

As described in Fig. 1, the motion picture retrieving 
system includes: a first motion picture shape descriptor 
abstracting unit 130, a motion picture retrieving device 
110, a motion picture database (DB) 120 and a motion 
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picture, shape descriptor metadata DB 150. The motion 
picture retrieving device 110 includes a second motion 
picture shape descriptor abstracting unit 130a, a motion 
picture shape descriptor similarity computing unit . Ill and 
5 a distance-based classification unit 112. 

Hereinafter, the operation^ of each element will - be 
described. . 

When segmented motion, picture 120 is inputted by a 
user, the * motion picture shape descriptors for the 

10 segmented motion picture 120 are abstracted. The 
abstracted motion picture shape descriptors are inputted to 
the motion picture shape descriptor similarity computing 
unit of the motion picture retrieving device 110. 

The motion picture stored in the motion picture DB 120 

15 for storing motion pictures is inputted to the second 
motion picture shape descriptor abstracting unit 130a in 
the motion picture retrieving device 110. Then, the 
information outputted from the second motion picture shape 
descriptor abstracting unit 130a is stored in the motion 

20 picture shape descriptor metadata DB 150 in the form of 
metadata. The motion picture shape descriptor similarity 
computing unit 111 calculates the difference (i.e., 
similarity) between the motion picture shape descriptors 
outputted from the first motion picture shape descriptor 

25 abstracting unit 130 and the motion picture shape 
descriptors in the motion picture shape descriptor metadata 
DB 150. To calculate the similarity (i.e., distance), a 
method using Euclidian distance which measures the distance 
between two vectors or a method using the sum of absolute 

30 difference is used. The distance-based classification unit 
112 sorts out the calculated distance information in the 
order of distances from close to far, abstracts 
corresponding metadata information from the motion picture 
shape descriptor metadata DB 150, and outputs the 

35 abstracted similar motion picture information 14 0 to the 
user. 
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Fig. 2 is a block diagram illustrating the motion 
picture shape descriptor abstracter of Fig. 1 in accordance 
with the embodiment of the - present invention. As 
illustrated in the drawing, the motion picture shape 
5 descriptor abstracting unit 230 of the present invention 
includes: a motion picture segmentation unit 210, a motion 
picture shape descriptor abstracting unit 230 and a motion 
picture metadata DB 250. The motion * picture shape 
descriptor abstracting unit 230 includes a shape abstracter 

10 231, a shape vector descriptor abstracter 233 and a 
statistical shape vector descriptor abstracter 235. 

Hereinafter, the operation of each element will be 
described. First, a motion picture 200 is inputted to the 
motion picture segmentation unit 210 and segmented 

15 temporally. The temporally segmented motion picture 2 00 is 
inputted to the shape abstracter 231, which then outputs 
shape information motion picture 232, corresponding to* one 
object. The shape information of each still picture of the 
shape information motion picture 232 is inputted to the 

20 shape vector descriptor abstracter 233^ which outputs a 
shape vector descriptor sequence 234. 

The shape vector descriptor sequence 234 outputted 
from the shape vector descriptor abstracter 233 is inputted 
to the statistical shape vector descriptor abstracter 235. 

25 The statistical shape vector descriptor abstracter 235 
outputs a motion picture shape descriptor 240 eventually, 
by using each or a combination of the Equations 1 through 5, 
each of which corresponds to the above enumerated (1) mean 
shape descriptor, (2) variance shape descriptor, (3) 

30 standard deviation shape descriptor and (4) differential 
shape descriptor. The motion picture shape descriptor 240 
is stored in the motion picture metadata DB 250 for storing 
motion picture metadata. 

Fig. 3 is a table showing the metadata stored in a 

35 motion picture metadata database for storing motion picture 
shape descriptors in accordance with the embodiment of the 



8 



wo 03/056463 




PCT/KR02/01830 



present invention- The metadata are classified based on 
motion picture shape vector descriptor, motion picture 
title, location of file, and the location of starting time 
in the original motion picture. 
5 Fig. 4 is a flow chart describing a method for 

abstracting a motion picture- shape descriptor in accordance, 
with the embodiment of the present invention. As shown in 
the drawing, to abstract a motion picture shape descriptor, 
at step S403, an input motion picture 400 is segmented 

10 temporally, and at step S405, a shape information of motion 
picture corresponding, to one object is abstracted from the 
temporally segmented motion picture. 

Subsequently, at step S407, a shape vector descriptor 
sequence is abstracted ■ from the abstracted shape 

15 information of motion picture. At step S409, a motion 
picture shape descriptor, which is a statistical shape 
descriptor, is abstracted from the shape vector descriptor 
sequence. Then, at step S411, the abstracted motion 
picture shape descriptor is stored in the motion picture 

20 metadata DB for storing motion picture metadata. 

As described above, the technology of the present 
invention can store the changing shape information of a 
motion picture object effectively by using a motion picture 
shape descriptor, and using the stored motion picture 

25 information for retrieving motion picture and, further, for 
video indexing. 

While the present invention has been described with 
respect to certain preferred embodiments, it will be 
apparent to those skilled in the art that various changes 

30 and modifications may be made withput departing from the 
scope of the invention as defined in the following claims. 
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What is claimed is; 

1. A system for retrieving motion picture, 
comprising: 

a motion picture segmentation means for segmenting 
motion picture temporally; 

a motion picture shape descriptor abstracting means 
for abstracting a motion picture shape descriptor from the 
segmented motion picture; and 

a motion picture metadata storing means for storing 
the motion picture shape descriptor as metadata. 

2. The system as recited in claim 1, wherein the 
motion picture shape descriptor abstracting means includes: 

15 a shape abstracting means for abstracting shape 

information corresponding to one object from the segmented 
motion picture; 

a shape vector descriptor abstracting means for 
abstracting shape vector descriptor sequence from the shape 
20 information; and 

a statistical shape vector descriptor abstracting 
means for abstracting a motion picture shape descriptor 
from the shape vector descriptor sequence. 

^5 3. The system as recited in claim 2, wherein the 

statistical shape vector descriptor abstracting means 
abstracts motion picture shape descriptor by using one or 
combination of a mean shape descriptor, a variance shape 
descriptor, a standard deviation shape descriptor and a 

30 differential shape -descriptor. 

4. The system as recited in claim 3, wherein the 
mean shape descriptor is obtained based on an Equation as: 

35 sd^"{m) = (Zi=iton sdi(m))/n , 
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wherein sdi = {sdi(l), sdi{2), sdi(3),.-., sdi (m) } . 

5. The system as recited in claim 3, wherein the 
variance shape descriptor is obtained based on an Equation 

5 as : 

V 

sd^^^(m) = (Ei=i to n- (sdi(m)- sd^^(m) )2)/n/(n-l) , 

wherein sd^"" (m) = (Si=i to n sdi(m))/n, and sdi = {sdi(l), 
10 sdi (2), sdi (3),..., sdi(m)}.. 

6. The system as recited in claim 3, wherein the 
standard deviation shape descriptor is obtained based on an 
Equation as : 

15 

sd^'^^(m) = sqrt(2i=iton (sdi(m)- sd^^ (m) ) 2) /n/ (n-1) , 

wherein sd^"" (m) = (Ei=i to n sdi(m))/n, and sdi (m) = 
{sdi(l), sdi(2), sdi(3),..., sdi(m)}. 
20 . 

7. The system as recited in claim 3, wherein the 
differential shape descriptor is obtained based on an 
Equation as: 

25 dsdr = (Sdr^-i * Pr+l) (Sdr * Pr) , 

wherein sdr denotes a shape descriptor abstracted from 
the m^^ shape information Sr; 

r is in the range of 0 < r < n; and 
30 Pr denotes a weight of the r^^ shape descriptor sdr- 

8. A system for retrieving motion picture, 
comprising: 

a first motion picture shape descriptor abstracting 
35 means for abstracting a first motion picture shape 
descriptors for motion picture; 
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a motion picture storing means for storing the motion 
picture; " 

a motion picture shape descriptor metadata storing 
means for storing the first motion picture shape 
descriptor; and 

a motion picture retrieving means- for calculating the 
similarity between the first motion picture shape 
descriptor abstracted from the motion picture shape 
descriptor abstracting means and a second motion picture 
shape descriptor outputted from the . motion picture shape 
descriptor metadata storing means, arranging the motion 
picture shape descriptor in the order of similarity from 
small to large, and outputting similar motion pictures. 

^- The system as recited in claim 8, wherein the 
motion picture retrieving means includes: 

a second motion picture shape descriptor abstracting 
means for abstracting motion picture shape descriptor from 
the. motion picture outputted from the motion picture 
storing means and storing the abstracted motion picture 
shape descriptor in the motion picture shape descriptor 
metadata storing means; 

a motion picture shape descriptor . similarity 
computing means for calculating the similarity between a 
first motion picture shape descriptor outputted from the 
first motion picture shape descriptor abstracting means and 
the second motion picture shape descriptor outputted from 
the motion picture shape descriptor metadata storing means; 
and 

a distance-based classification means ' for classifying- 
the similarity outputted from the motion picture shape 
descriptor similarity computing means and outputting the 
similar motion pictures. 

The system as recited in claim 9, wherein the 
distance-based classification means classifies ' the 



20 



25 



30 



12 



wo 03/056463 




PCT/KR02/01830 



similarity in the order of distance from close to far. 

11. The system as recited in claim 9, wherein the 
motion picture shape descriptor similarity computing means 

5 computes the similarity based on an Euclidian distance 
between two input information vectors, or a sum of absolute 
differences, - 

12. A method for abstracting a motion picture shape 
10 descriptor having statistical characteristics of still 

picture shape descriptors to be applied to a motion picture 
shape descriptor abstracting apparatus, the method 
comprising the steps of: 

a) segmenting a motion picture temporally and 
15 abstracting shape information corresponding to one object 

from the temporally segmented motion picture; 

b) abstracting a motion picture shape descriptor, 
which is a statistical shape vector descriptor, from the 
shape information; and 

20 c) storing the motion picture shape descriptor in a 

motion picture metadata storing means. 

13. The method as recited in claim 12, further 
comprising the steps of: 

25 d) abstracting a shape vector descriptor sequence 

from the abstracted shape information of motion picture in 
order to abstract the motion picture shape descriptor; and 

e) abstracting a motion picture shape descriptor, 
which is a statistical shape vector descriptor, from the 

30 shape vector descriptor sequence . 

14. The method as recited in claim 13, wherein the 
motion picture shape descriptor, which is a statistical 
shape vector descriptor of the step e) , can be obtained 

35 based on an Equation as: 



13 



wo 03/056463 PCT/KR02/01830 



sd^"(m) = (Ei=iton sdi(m))/n , 

wherein sdi = {sdi(l), sdi{2)^ sdi(3),..:, sdi (m) } . 

5 15. The method as recited in claim 13, wherein the 

motion picture shape descriptor can be obtained based on an 
Equation as-: 

sd^^Mm) (Si^iton (sdi(m)- sd^^(m))")/n/(n-l) , 

wherein sd^"" (m) = {Ei=i to n sdi(m))/n^ and sdi = {sdi(l), 
sdi (2), sdi (3 ),..., sdi(m)}. 

16. The method as recited in claim 13, wherein the 
15 motion picture shape descriptor can be obtained based on an 

Equation as: 

sd^"(m) - sqrt(Ei«iton (sdi(m)- sd^^ (m) ) 2) / (n-1) , 

20 wherein sd^^ (m) = (Ei=i to n sdi (m) ) /n, and sdi (m) = 

{sdi(l), sdi(2), sdi(3),..., sdi (m) } . 

17. The method as recited in claim 13, wherein the 
motion picture shape descriptor can be obtained based on an 

25 Equation as.: 

dsdr - (Sdr+l * Pr+l) (sdr * Pr) . 

wherein sdr denotes a shape descriptor abstracted from 
30 the m^^ shape information Sr; 

r is in the range of 0 < r < n; and 

Pr denotes a weight of the r^.^ shape descriptor sdr. 

18- A computer-based recording medium for recording 
35 a program for executing a method for abstracting motion 
picture shape descriptors, the method comprising the steps 

14 
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of: 

a) segmenting a motion picture temporally and 
abstracting shape infbrmat'lon corresponding to one object 
from the temporally segmented motion picture; 
5 b). abstracting a motion picture shape descriptor, 

which is a statistical shape vector descriptor, from the 
shape information; and 

c) storing the motion picture shape descriptor ' in a 
motion picture metadata storing means, 
10 wherein the program is implemented in a • motion 

picture shape descriptor abstracting apparatus provided 
with a processor. 
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